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Abstract 



We show that a certain representation of the matrix-product can be computed with 
7jo(i) multipUcations. We also show, that siumilar representations of matrices can be 
compressed enormously. 

1 Introduction 

Gigantic matrices emerge in applications in numerous fields of mathematics, computer science, 
statistics and engineering. The storage and the basic linear-algebraic operations of these 
matrices is a difficult task which should be handled in data-mining, signal-processing, image- 
processing and various other applications day bay day. 

Here we describe a method which, by the simplest tools, may help in these tasks. We 
describe how to store large matrices or do linear algebraic operations on them if it is sufficient 
to retrieve the matrices or the results of the linear-algebraic operations only in a certain form, 
called 1-a-strong representation modulo 6 in |(Trofl3j . 

For n X n matrices A and B with elements from set {0, 1,2, 3, 4, 5}, our results include: 

• The computation of n°^^^ x n°^-^^ matrix A' with elements from set {0, 1, 2, 3, 4, 5}, such 
that from A' one can retrieve the 1-a-strong representation of the n x n matrix A; 
here both operations are simple linear transformations (here o(l) denotes a sequence of 
positive numbers going to while n goes to the infity). 

• The computation of the 1-a-strong representation of the matrix-product AB with n°^^^ 
multiplications, significantly improving our earlier result of computing the 1-a-strong 
representation of the matrix-product with n^+°(^) multiplications |Gro03j . 



1.1 Earlier Results: Matrix Product 

The matrix multiplication is a basic operation in mathematics in applications in almost 
every branch of mathematics itself, and also in the science and engineering in general. An 
important problem is finding algorithms for fast matrix multiplication. The natural algorithm 
for computing the product of two nx n matrices uses multiplications. The first, surprising 
algorithm for fast matrix multiplication was the recursive method of Strassen |Str69j . with 
0^^2.81-^ multiplications. After a long line of results, the best known algorithm today was 
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given by Coppersmith and Winograd |CW90j . requiring only 0(n^'^^^) multiplications. Some 
of these methods can be applied successfully in practice for the multiplication of large matrices 
|Bai88j . 

The best lower bounds for the number of needed multiplications are between 2.5v? and 
3n^, depending on the underlying fields (see |Bla99j . |Bsh89j . |Shp01| ). A celebrated result of 
Raz |H,azn2j is an logn) lower bound for the number of multiplications, if only bounded 
scalar multipliers can be used in the algorithm. 

In |Gro03j we gave an algorithm with n^+°(^) multiplications for computing a represen- 
tation of the matrix product modulo non-prime power composite numbers (e.g., 6). The 
algorithm was a straightforward application of a method of computing the representation of 
the dot-product of two length-n vectors with only n°^^^ multiplications. 

In the present work, we significantly improve the results of |Gro03j . we gave an algo- 
rithm for computing a representation of the product of two n x n matrices with only n"^^^ 
multiplications. 

1.1.1 Why do we count only the multiplications? 

In algebraic algorithms it is quite usual to count only the multiplications in a computation. 
The reason for this is that the multiplication is considered to be a harder operation than 
the addition in most practical applications, and moreover, the multiplication is proven to be 
harder in most theoretical models of computation. 

For example, computing the PARITY is reduced to computing the multiplication of two 
n-bit sequences, and, consequently, two n-bit sequences cannot be multiplied on a polynomial- 
size, constant-depth Boolean circuit |FSS84j . while it is well known, that two n-bit sequences 
can be added in such a circuit. 

Another example is the communication complexity |KN97j of the computation of multi- 
variable polynomials. Here two players, Alice and Bob, want to co-operatively compute the 
value of a 2n- variable polynomial f{x,y) modulo m, where the polynomial / is known for 
both players, while x is known only for Alice, and y is known only for Bob. If / is a linear 
polynomial, then they can compute f{x,y) with communicating only O(logm) bits: Indeed, 
if 

f{x,y) = ^ajXi + '^bjyj, 

i j 

then Alice communicates to Bob the value 

^ QiXi mod m, 

i 

then Bob will know the value of f{x,y). Similarly, if f{x,y) can be given as the sum of u 
products, each with v clauses: 

U V 

fix, y) = ^ YlYl O'ifiuXi + bjf^^yj, 

fj.=l u=l i,j 

then f{x,y) can be computed with 0{uvlogm) bits of communication. This example also 
shows that in communication complexity the multiplications may be harder than the addi- 
tions. Chor and Goldreich |CG85j proved that even approximating the dot-product function 

n 

f{x,y) = ^Xiyi 

i=l 
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needs bits of communication; consequently, multiplications are really hard in the com- 
munication model. 

1.2 Earlier Results: Matrix Storage 

There are lots of results concerning sparse matrix-storage and operations. Here we do not 
assume anything about the sparsity of our input matrices. 

We should mention here the works of Frieze and Kannan |FK99j and |FKV98j on approx- 
imations of large matrices with small rank matrices with very fast algorithms. 

2 Preliminaries 

First we need to review several definitions and statements from |Gro03j . 
2.1 A-strong representations 

In |(Trof)2j we gave the definition of the a-strong (i.e., alternative- strong) representation of 
polynomials. Here we define the alternative, and the 0-a-strong and the 1-a-strong represen- 
tations of polynomials. Note that the 0-a-strong representation, defined here, coincides with 
the a-strong representation of the paper Gro02;. 

Note also, that for prime or prime-power moduli, polynomials and their representations 
(defined below), coincide. Perhaps that is the reason that such representations were not 
defined before. 

Definition 1 Let m be a composite number m = p^^p^ • • • p^ ■ Let denote the ring of 
modulo m integers. Let f be a polynomial of n variables over Z^: 

f{xx,X2,...,Xn)= ^ aixi, 
/C{l,2,...,ra} 

where aj E Zm, xj = Hje/ Then we say that 

g{xi,X2,...,Xn)= ^ bixi, 

/C{l,2,...,n} 

is an 

• alternative representation of f modulo m, if 

V/C {l,2,...,n} 3j e {1,2,...,^} : ai = hi (mod pj^; 

• 0-a-strong representation of f modulo m, if it is an alternative representation, and, 
furthermore, if for some i, ai ^ 6/ (mod p^'), then bj = (mod p^'); 

• 1-a-strong representation of f modulo m, if it is an alternative representation, and, 
furthermore, if for some i, ai ^ 6/ (mod pf" ), then a/ = (mod m); 
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Example 2 Let m = 6, and let f{xi,X2,X3) = xiX2 + X2X^ + X1X3, then 



g{xi,X2, , X3) = 3xiX2 + 4X2X3 + X1X3 



is a 0-a-strong representation of f modulo 6; 



g{xi,X2, ,X3) = X1X2 + X2X3 + X1X3 + 3x1 + 



is a 1-a-strong representation of f modulo 6; 



g{xi,X2, , X3) = 3xiX2 + 4x2X3 + X1X3 + 3xi + 4x2 



is an alternative representation modulo 6. 

In other words, for modulus 6, in the alternative representation, each coefficient is correct 
either modulo 2 or modulo 3, but not necessarily both. 

In the 0-a-strong representation, the coefficients are always correct both modulo 2 and 
3, the non-zeroes are allowed to be correct either modulo 2 or 3, and if they are not correct 
modulo one of them, say 2, then they should be mod 2. 

In the 1-a-strong representation, the non-zero coefficients of / are correct for both moduli 
in g, but the zero coefficients of / can be non-zero either modulo 2 or modulo 3 in g, but not 
both. 

Example 3 Let m = 6. Then = xy — 3xy + 2xy is not a 1-a-strong representation of xy. 
Similarly, polynomial f + 2g + 3h is a mod 6 1-a-strong representation of polynomial f if 
and only if g and h do not have common monomials with f, and g does not have common 
monomials with h. 

2.2 Previous results for a-strong representations 

We considered elementary symmetric polynomials 



in |Gro02j . and proved that for constant /c's, 0-a-strong representations of elementary sym- 
metric polynomials can be computed dramatically faster over non-prime-power composites 
than over primes: we gave a depth-3 multilinear arithmetic circuit with sub-polynomial num- 
ber of multiplications (i.e., n^,Ve > 0), while over fields or prime moduli computing these 
polynomials on depth-3 multilinear circuits needs polynomial (i.e., n^^^^) multiplications. 

Here depth-3 multi-linear, homogeneous arithmetic circuits computes polynomials of the 
form 



<k 



E n 



n 



/C{l,2,...,n} ig/ 
/|=fe 




(1). 



j=l k=l i=l 



These circuits are sometimes called SHE circuits. 



In |(Tro02j . we proved the following theorem: 
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Theorem 4 ( |Gro02 | ) (i) Let m = piP2, where pi / p2 are primes. Then a degree-2 
0-a-strong representation of 

ije{l,2,...,n} 

modulo m: 

^ aijXiVj (2) 

i,je{l,2,...,n} 

can he computed on a bilinear SnS circuit of size 



exp(0(-\/lognloglogn)). 
Moreover, this representation satisfies that \/i ^ j : mj = aji . 

(a) Let the prime decomposition of m = p^^V^ ' ' ' Pr"^ ■ Then a degree-2 0-a-strong represen- 
tation of S^{x,y) modulo m of the form (1) can be computed on a bilinear TiUT, circuit 
of size 



exp ^ \J log n (log log n)''' ^ 
Moreover, this representation satisfies that \/i ^ j : aij = aji . 

□ 

Corollary 5 The 0-a-strong representation of (2) can be computed using 



exp (O ( y'log n log log n) ) 

multiplications. 

The following result is the basis of our theorems in the present paper. 

Theorem 6 ( jGro03j ) (i) Let m = piP2, where pi ^ p2 are primes. Then a degree-2 
1-a-strong representation of the dot-product 



f{xi,X2, ■ ■ ■,Xn,yi,y2, ■■■,yn) = '^Xiyi 



1=1 



can be computed with 



exp(0(-\/log n log log n)) (3) 

multiplications. 

(a) Let the prime decomposition of m = p^iP^2 '"VV^ ■ Then a degree-2 1-a-strong repre- 
sentation of the dot-product f modulo m can be computed using 



exp ( O ( {/log n(log log n)"-^ ) ) (4) 

multiplications. 
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(in) Moreover, the representations of (i) and (ii) can he computed on bilinear SnS circuits 
of size (3), and (4), respectively. 

We reproduce here the short proof: 
Proof: Let g(x, y) = g{xi,X2, . . . , Xn, yi,y2, ■ ■ ■ , Un) be the degree-2 polynomial from The- 
oremlHlwhich is a 0-a-strong representation of S'^{x,y). Then consider polynomial 

h{x, y) = {xi + X2 + . . . + xn){yi + y2 + ■ ■ ■ + yn) - g{x, y) (5*). 

In h{x,y), the coefficients of monomials Xiyi are all I's modulo m, and the coefficients of 
monomials Xjj/j, for i j are at least for one prime-power divisor of m. Consequently, by 
Definition^ h{x,y) is a 1-a-strong representation of the dot-product f{x,y). □ 

The following definition is a natural generalization of the a-strong representations for 
matrices: 

Definition 7 Let A = {oij} and B = {hij} he two nxn matrices over Z^- Then C = {cjj} is 
the alternative (1-a-strong, 0-a-strong) representation of the matrix A if for i < i, j < n, the 
polynomial oij of variables {ajj} is an alternative (1-a-strong, 0-a-strong) representation 
of polynomial Oij . 

Consequently, we say that matrix D = {dij} is an alternative (1-a-strong, 0-a-strong) 
representation of the product-matrix AB, if for 1 < i, j < n, dij is an alternative (1-a-strong, 
0-a-strong) representation of polynomial 

n 

fc=l 

modulo m, respectively. 

In [(irronSj we proved by applications of Theorem El that the 1-a-strong representations 
modulo 6 of the product of two nxn can be computed with n'^~^"^^^ multiplications. Here 
we significantly improve this result: we show that the 1-a-strong representation can be com- 
puted by multiplications. Moreover, the computation can be performed on a depth-3 
homogeneous, bi-linear SIIS circuit (bi-linear means that £ = 2 in (1)). 

Let us stress that we emphasize that the computation can be performed on such a circuit 
because this circuit is perhaps the simplest model of computation in which matrix-product 
can be computed; the main result is that a certain representation of the matrix product can 
be computed by so few multiplications. 



3 Our result for matrix compression 

For simplicity, we prove our result here only for modulus 6; for other moduli, the poof is very 
similar, using Theorem El part (ii) instead of part (i). 

If we do not say otherwise, the computations are modulo 6. 

By Theorem El a 1-a-strong representation of the dot-product J27=i ^iVi can be computed 

as 

n t / n \ / n \ 

^ Xiyi + 3fi((x, y) + Ah{x, v) = ^\ Y1 ^ij^i ^^^y^ (*^) 

i=l j=l \i=l ) \i=\ ) 
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where bij,Cij £ {0, 1} and where both g and h has the following form: J2ij^j o-ij^iVj, aij mod 
6 € {0, 1}, and no term Xii/j appears in both / and g; and t = exp{0{^/^og/i^logTogn)) = n°^^\ 
Now, let us observe that for each j = 1,2, . . . ,t, 

n 
1=1 

is a linear combination of variables Xj. If we plug in vectors instead of just variables in these 
homogeneous linear forms, then we will get linear combinations of the vectors. 

Now, let X denote an n x n matrix with entries as variables {xij}, and let Xi denote its 
i^^ column, for i = 1, 2, . . . , n and let B = {bij}, an n x t matrix of (*1) and let C = {cij} an 
n X t matrix of (*1). 

The columns of the n x t matrix-product 

XB 

gives the linear combinations specified by (*2). However, 

XBC^ 

is an n X n matrix, such that its column i/ is equal to 

x^ + 3g^{X) + AK{X) 

where guiX) and hnu{X) are linear combinations of the columns of X such that none of 
which contains Xi, and they do not contain the same column with non-zero coefficients. The 
proof of this fact is obvious from (*1), observing that plugging in 

y'^ = (0,0,...,"l",0,...,0) 

we simply generate some linear combinations of the columns of matrix XB, and the coefficient 
in these combinations are nothing else that the rows of C. 

Consequently, we proved the following implication of Theorem 1^1 

Theorem 8 For any non-prime-power m > 1 and for the n x n matrix X = {xij}, there 
exist effectively computable constant n x t matrices B and C , such that 

XBC^ 

is a 1-a-strong representation of matrix X modulo m, where t is equal to quantity (4), that 
is, t = n°^^\ Moreover, every column ofXBC^ is a linear combination of the columns of X . 

The dimension-defying implication of Theorem |H1 is that X is an n x n matrix, XB is an 
n X n"(^) matrix, and XBC^ is again an n x n matrix. 
An easy corollary of Theorem |H1 that 

Corollary 9 With the notations of Theorem CB^X is a 1-a-strong representation of 
matrix X modulo m, where t is equal to quantity (4), that is, t = n°^^\ Moreover, every row 
ofCB^X is a linear combination of the rows of X. 
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Our main result in this section is the following 

Theorem 10 For any non-prime-power m > 1 and for the n x n matrix X = {xij}, there 
exist effectively computable constant n x t matrices B and C , such that 

b'^xb 

is a t X t matrix, where t is equal to quantity (4), that is, t = n°^^\ and matrix 

cb'^xbc^ 

is a 1-a-strong representation of matrix X modulo m. 

The dimension-defying implication of Theorem is that from the n x n matrix X with 
simple linear transformations we make the tiny n°^^^ x n°^^^ matrix B^XB, and from this, 
again with simple linear transformations, n x n matrix CB^XBC^ , where it is a 1-a-strong 
representation of matrix X modulo m. 

Proof: Prom Theorem |S1 XBC^ is a 1-a-strong representation of matrix X modulo m. 
Moreover, every column of XBC^ is a linear combination of the columns of X. By Corollary 
1^1 for any n x nY, CB^Y is a 1-a-strong representation of matrix Y modulo m, and, every 
row of CB^Y is a linear combination of the rows of Y. Plugging in y = XBC^, we get that 
the matrix CB^XBC^ is a 1-a-strong representation of the matrix XBC^, moreover, its 
rows are linear combinations of the rows of XBC'^, that is, CB^XBC^ is also a 1-a-strong 
representation of the original X.O 

a 

4 Our result for matrix multiplication 

4.1 Preliminaries 

We need to define a sort of generalization of matrix-product: 

Definition 11 / : R^^ R is a homogeneous bilinear function over ring R if 

f{xi,X2,...,Xn,yi,y2,---,yn) = ^ij^iVj 

l<i,j<n 

for some aij E R. Let U = {uij} be an u x n matrix over ring R, and let V = {vke} be an 
n X V matrix over R. Then 

U{f)V 

denotes the u x v matrix over R with entries vun, where 

Wil = f{Uii,Ui2, . . . ,Uin,Vu,V2i, ■ ■ ■ ,Vnt)- 

Pirst we need a simple lemma, stating that the associativity of the matrix multiplication 
is satisfied also for the "strange" matrix-multiplication defined in Definition 1111 
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Lemma 12 Letf{xi,X2, . . . ,Xn,yi,y2, ■ ■ ■ ,yn) = J2i<i,j<n"-ijXiyj andg{xi,X2, ■ ■ ■ ,x„,yi,y2, 
X^Ki j<v bijXiyj be homogeneous bilinear functions over the ring R. Let U = {uij} be an uxn 
matrix, and let V = {vke} be an n x v matrix, and W = {wij} be a v x w matrix over R, 
where u, n, w are positive integers. Then 

{U{f)V) [g)W = Uif) iV{g)W) , 

that is, the "strange" matrix-multiplication, given in Dehnition MlV is associative. 

Proof 1: The proof is obvious from the homogeneous bi-hnearity of / and g. 

Proof 2: We also give a more detailed proof for the lemma. The entry of row i and column 

k of matrix U{f)V can be written as 

z,t 

Consequently, the entry in row i and column r of {U{f)V){g)W is 

"^bkl ^a^tUizVtk Wfr- 

k,e \ z,t ) 
On the other hand, entry (t, r) in V{g)W is 

^bktVtkWllr, 
k,l 

and entry (i,r) in U{f){y{g)W) is 

X] 0.ztUiz ^ bkeVtkWir, 

z,t k,e 

and this proves our statement. □ 

Now we are in the position of stating and proving our main theorem for matrix multipli- 
cations: 

Theorem 13 Let X and Y two uxn matrices, and let m > 1 be a non-prime-power integer. 
Then the 1-a-strong representation of the matrix-product XY can be computed with t = n"'^^) 
multiplication, where t is given by (4). 

Proof: We use Theorem |H1 and Corollary |3 Let us consider t x n matrix B^X and t x n 
matrix YB; these matrices can be computed without any multiplications from X and Y (we 
do not count multiplications by constants). Let h{x, y) be the homogeneous bi-linear function 
(5*). Then 

B'^X{h)YB 

can be computed with n°^^^ multiplications (Note, that because of Lemma ll2l the associativity 
holds). Now compute matrix 

CB^X{f)YBC^ = {CB^Y){f){YBC^) 

without any further (non-constant) multiplication. By Theorem |H1 and Corollary |UJ CB^X 
and YBC^ is a 1-a-strong representations of X and Y respectively, and they are the linear 
combinations of the rows of X and columns of Y , respectively. Consequently, using Theorem 
El CB^X{f)YBC^ is a 1-a-strong representation of XY.no 
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